Search for: All records

The interaction and collaboration between humans and multiple robots represent a novel field of research known as human multirobot systems. Adequately designed systems within this field allow teams composed of both humans and robots to work together effectively on tasks, such as monitoring, exploration, and search and rescue operations. This article presents a deep reinforcement learning-based affective workload allocation controller specifically for multihuman multirobot teams. The proposed controller can dynamically reallocate workloads based on the performance of the operators during collaborative missions with multirobot systems. The operators' performances are evaluated through the scores of a self-reported questionnaire (i.e., subjective measurement) and the results of a deep learning-based cognitive workload prediction algorithm that uses physiological and behavioral data (i.e., objective measurement). To evaluate the effectiveness of the proposed controller, we conduct an exploratory user experiment with various allocation strategies. The user experiment uses a multihuman multirobot CCTV monitoring task as an example and carry out comprehensive real-world experiments with 32 human subjects for both quantitative measurement and qualitative analysis. Our results demonstrate the performance and effectiveness of the proposed controller and highlight the importance of incorporating both subjective and objective measurements of the operators' cognitive workload as well as seeking consent for workload transitions, to enhance the performance of multihuman multirobot teams. 

This paper presents MOCAS, a multimodal dataset dedicated for human cognitive workload (CWL) assessment. In contrast to existing datasets based on virtual game stimuli, the data in MOCAS was collected from realistic closed-circuit television (CCTV) monitoring tasks, increasing its applicability for real-world scenarios. To build MOCAS, two off-the-shelf wearable sensors and one webcam were utilized to collect physiological signals and behavioral features from 21 human subjects. After each task, participants reported their CWL by completing the NASA-Task Load Index (NASA-TLX) and Instantaneous Self-Assessment (ISA). Personal background (e.g., personality and prior experience) was surveyed using demographic and Big Five Factor personality questionnaires, and two domains of subjective emotion information (i.e., arousal and valence) were obtained from the Self-Assessment Manikin (SAM), which could serve as potential indicators for improving CWL recognition performance. Technical validation was conducted to demonstrate that target CWL levels were elicited during simultaneous CCTV monitoring tasks; its results support the high quality of the collected multimodal signals. 

Large atmospheric boundary layer fluctuations and smaller turbine-scale vorticity dynamics are separately hypothesized to initiate the wind turbine wake meandering phenomenon, a coherent, dynamic, turbine-scale oscillation of the far wake. Triadic interactions, the mechanism of energy transfers between scales, manifest as triples of wavenumbers or frequencies and can be characterized through bispectral analyses. The bispectrum, which correlates the two frequencies to their sum, is calculated by two recently developed multi-dimensional modal decomposition methods: scale-specific energy transfer method and bispectral mode decomposition. Large-eddy simulation of a utility-scale wind turbine in an atmospheric boundary layer with a broad range of large length-scales is used to acquire instantaneous velocity snapshots. The bispectrum from both methods identifies prominent upwind and wake meandering interactions that create a broad range of energy scales including the wake meandering scale. The coherent kinetic energy associated with the interactions shows strong correlation between upwind scales and wake meandering. 

Turbulent wakes are often characterized by dominant coherent structures over disparate scales. Dynamics of their behaviour can be attributed to interscale energy dynamics and triadic interactions. We develop a methodology to quantify the dynamics of kinetic energy of specific scales. Coherent motions are characterized by the triple decomposition and used to define mean, coherent and random velocity. Specific scales of coherent structures are identified through dynamic mode decomposition, whereby the total coherent velocity is separated into a set of velocities classified by frequency. The coherent kinetic energy of a specific scale is defined by a frequency triad of scale-specific velocities. Equations for the balance of scale-specific coherent kinetic energy are derived to interpret interscale dynamics. The methodology is demonstrated on three wake flows: (i)$${Re}=175$$flow over a cylinder; (ii) a direct numerical simulation of$${Re}=3900$$flow over a cylinder; and (iii) a large-eddy simulation of a utility-scale wind turbine. The cylinder wake cases show that energy transfer starts with vortex shedding and redistributes energy through resonance of higher harmonics. The scale-specific coherent kinetic energy balance quantifies the distribution of transport and transfer among coherent, mean and random scales. The coherent kinetic energy in the rotor scales and the hub vortex scale in the wind turbine interact to produce new scales. The analysis reveals that vortices at the blade root interact with the hub vortex formed behind the nacelle, which has implications for the proliferation of scales in the downwind near wake.